Internet enabled clients, such as smart phones, personal computers, tablets, gaming systems and the like have become prevalent in recent years. Given the proliferation of Internet enabled clients and far-reaching Internet access, more and more users access online content hosted by servers. The vast majority of users access online content from hosts for legitimate reasons. However, there are illegitimate users who try to take down the hosts of online content with malicious clients, whether to simply deny services to other users or for more nefarious purposes.
An example of online content is web applications, which are made available for clients to access by a host. Web applications have become a popular method for providing online functionality to clients. Hundreds of thousands of web applications have been deployed and many more are offered to provide new functionality to clients over a network. Many of these web applications have vulnerabilities, which may be exploited to varying degrees by malicious clients. For example, a malicious client may exploit a vulnerability of a web application to bring down the web application/host, or worse yet, breach an organization, obtain valuable information, and utilize or expose the information for nefarious purposes. Given the ubiquity of web applications, they often provide an easy entry point for hackers to exploit with a malicious client and set a foothold within a target organization via the host of the web application to obtain valuable information.
Traditional methods of protecting web applications comprise a web application firewall (WAF). One well-known example of a WAF is MOD SECURITY, an open source project that has its roots as an Apache Webserver module. Many other commercially available WAFs exist as well. WAFs can be categorized as follows: Negative Security Model (signature based), Positive Security Model (signature based), and Profiling/Anomaly Detection (unsupervised).
A common issue with traditional signature based WAFs is that they are difficult to deploy and maintain to stay current with existing and emerging threats. While deploying a WAF with a basic rule set provides some level of protection, there are very well documented methods for evading traditional, signature based WAFs. In many cases, signature based WAFs are also not adequately tuned due to the lack of skilled operators available. Further, as signature based WAF implementations grow in complexity with the number of rules implemented, a web application behind the WAF will often experience performance issues. This occurs because negative and positive security models are based on signatures that are written as regular expressions (e.g., Regex's), that are computationally expensive and have to be executed for every HTTP(s) request seen by the host in order to filter traffic.
Profiling/anomaly detection WAFs attempt to learn patterns in web application traffic by passively inspecting HTTP(s) requests to determine malicious behavior. A profile is built based on either static source code analysis or probing of the web application in a staging environment. During deployment, requests are checked against the profile and backend database access to identify whether they are attempting to exploit a known vulnerability. A major drawback of this approach is that profiles extend poorly to general and new attack types, as they only capture attacks on specific known vulnerabilities, such as SQL Injection, for example. Accordingly, profiling/anomaly detection WAFs often require an inordinate amount of time to determine which behaviors are malicious unless the behavior meets specific criteria.
Additionally, and more generally, traditional WAFs are ineffective at providing a good balance between user experience and web application protection as they generate far too many false positives. A false positive is the miss categorization of a non-malicious client and/or behavior as malicious. From a user experience perspective, the miss categorization of the user's client or behavior as malicious negatively impacts the user's experience when engaging the web application (e.g., access is restricted and/or must pass CAPTCHAs and other challenges). In terms of web application protection, a high number of false positives results in a noisy alerting system that an administrator must constantly monitor to rectify false positives in order to avoid blockage of legitimate user traffic. Further, given the lack of skilled operators, a high number of false positives in a noisy alert system increases difficulty on administrators to distinguish between legitimate and non-legitimate traffic.